System and method for improving video and other media playback

ABSTRACT

Techniques for efficiently capturing, generating and distributing more realistic images and video are provided. Among other aspects of the invention, a system is provided that includes tags that display dynamic image elements on a display with the aid of a physics model. In other aspects, the system selects among available physics models and virtual objects that serve as potential subjects for rendering dynamic image elements, based on system resource costs and resolution standards. In still other aspects, a new image file format contains embedded physical model and virtual object data for rendering images and video.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation-in-part of U.S. application Ser. No.13/329,190, filed Dec. 16, 2011. This application also is acontinuation-in-part of U.S. application Ser. No. 13/326,235, filed Dec.14, 2011. The entire contents and disclosures of both of these patentapplications are incorporated by reference herein in their entirety.

FIELD OF THE INVENTION

The present invention relates to the field of report, video and imagegeneration. The present invention also relates to information databasesand physical models.

BACKGROUND

In the visual arts, a series of related images or reports may be usedfor a variety of purposes. For example, in the motion picture industry,a series of related images taken at time intervals are later projectedto generate the illusion of motion to a viewer. In the field offinancial reporting, a series of related images (such as analyticalreports) may be required to depict financial changes in a particularindustry or company, or changes from one company to another or from onearea to another. Regardless of the field, traditionally, a new image ofcomparable or similar size may be recorded and projected for certainintervals taken.

In the field of photography, the detection and treatment of shapeswithin an image or series of images has been sought. For example,certain technology by Sony Corporation permits the recognition of humanfaces and smiles by cameras. Also in the motion picture industry, theprojection of background images, further away from a foreground subject,into composite images is possible. This is often accomplished by use ofa backdrop color (as in “green-screen” technology) which a system isable to replace with a desired background image, which generally doesnot interfere with the foreground subject in a recorded composite imageincluding the two.

More recently, augmented reality systems have been developed, whichpermit the synthesis of computer generated (“CG”) images with capturedimages from photographic sources, with the aid of marker objectsrecognized by the system which are used as a location point forprojecting a computer-generated image. These systems result in thegeneration of composite images with both real-world and CG elements.

The present invention relates to techniques, systems and methods formore efficiently and more richly capturing, recording, transmitting,projecting and viewing both fixed and dynamic elements, image aspects orcomponents (which terms can be used interchangeably in this application)of related images. The present invention also relates to new imageformats, that may not have intervals or gaps and that may not rely onthe persistence of vision for the appearance motion.

Generally speaking, images, sound and other sensed phenomena have beencaptured in digital formats and media, which have had several advantagesover analog storage formats and media, including relative immunity todegradation and easy computation by digital computers. However, digitalformats have the disadvantage of fixed and uniform stepped intervals,which noticeably reduces the quality of playback, projection andexperience, particularly at high frequencies of sound or light waves,because higher frequency waves are thereby expressed with a moreincomplete waveform shape than lower frequencies. In response, severaldigital-to-analog converters have been introduced into the market, whichattempt to “smooth” the stepped intervals with interpolated data betweenthe bits of digital data. But digital-to-analog conversion causes datadegradation and, in any event, can never accurately restore the originalform of the phenomenon recorded.

SUMMARY OF THE INVENTION

Techniques for generating images quickly, efficiently and with greaterverisimilitude are provided. A preferred embodiment uses computerhardware for capturing and generating images with dynamic and fixedrepresentational elements (such as picture and text), metadatacomponents and Relational Aspects (defined further below) with otherelements, which may be associated with identified data of a database.The data and Relational Aspects may themselves be dynamic, and the data,Relational Aspects and representational elements may be dynamic inseries. In certain aspects of the invention, dynamic elements ofgenerated images, or their CG 3-D subjects, may be descriptively orphysically modeled, and may reposition or otherwise affect otherelements of the generated images, for example, at particular dynamiccoordinates, at the user's direction and according to differences in thedynamic elements over time. Certain elements, such as changes intexture, lighting and other representational or other data and aspectsmay be recorded and tolerated (in fixed element detection) and exerted(in generation onto fixed and dynamic elements in the images generated),and may involve Element Seeding or Borrowing (defined further below) andcontinuous changes throughout an image stream, for example, as definedby analog media, calculus derivatives and integrals or vectors.Management of the system may be by user interface which may entaildynamic templating with database and Relational Aspect creation andassociation, fixed and dynamic tagging, component selection andpositioning, 3-dimensional (“3-D”) and 2-dimensional (“2-D”) element,affector and subject placement and definition and treatment, and otherimage generation and capture parameters.

CANONS OF CONSTRUCTION AND DEFINITIONS

Where any term is set forth in a sentence, clause or statement(“statement”), each possible meaning, significance and/or sense of anyterm used in this application should be read as if separately,conjunctively and/or alternatively set forth in additional statements,as necessary to exhaust the possible meanings of each such term and eachsuch statement.

It should also be understood that, for convenience and readability, thisapplication may set forth particular pronouns and other linguisticqualifiers of various specific gender and number, but, where thisoccurs, all other logically possible gender and number alternativesshould also be read in as both conjunctive and alternative statements,as if equally, separately set forth therein.

Within the context of this application, unless otherwise indicated, thefollowing terms have the specific meaning described herein:

“Image” means a visual or other representation or communicationinvolving, at least in part, a tangible medium, where it is transmittedand/or recorded (and also refers to the recording itself), and does notnecessarily, but may, include associated non-representational orpartially representational elements, such as metadata and internal andexternal Relational Aspects, (for example, Seed or Borrowed Elements,defined further in this application, below, and such as an imageboundary deformation characteristic and acceleration second derivativeof position of a subject object, respectively). Images may be 2-,3-dimensional (“2-D” or “3-D”) or otherwise multidimensional and mayrefer to composites of other images and non-visual phenomena, such asother electromagnetic radiation, sound waves, olfactory, or tactilemedia. Thus, in addition to traditional visual images, an “image,” asmeant in this application, may refer to representations, communications,transmissions and recordings that may or may not be rendered depictedvisually, such as a sound, smell, UV photograph or 3-dimensional tactilerepresentation. When used in this application and its claims, this andevery other term defined in this application means each and every senseor part of a sense individually, together and in any combination, as ifso set forth from the text of the definitions, such as this, in thedefinition sections of this application.“Dynamic Element” means a part (including, but not limited to, any size,parameter, shape, contour, location, sound, timbre, color, temperature,filter, vibration, tone, effect, texture, luminosity, electromagneticradiation spectra, modulation or other emission or other aspect of thatpart), component, attribute, characteristic, aspect, filter or othereffect of, or other thing substantially impacting or affecting, an imagethat may change over the course of a function of the image or imagestream or from image-to-image in a series of images (or part of such aseries) of which the image is a part, or that may change as a result ofdata changing in an associated database. In addition to RepresentationalDynamic Elements, Dynamic Elements may include Non-RepresentationalElements, such as metadata and Relational Aspects, as further defined inthis application. Image Elements include their alternate storage andtransmission forms, for example as encoded signals and/or other storageor transmission media and formats, that may require furthermanipulation, translation, rendering, projection or other treatment tobe experienced by a user as a part of an image, as further defined inthis application.“Dynamic Representational Element” means a Dynamic Element, as furtherdefined in this application, which relates to or affects therepresentation or representational nature of an image or understandingor cognition by one experiencing the image or representation of theimage.“Fixed Element” means a part (including, but not limited to, any size,parameter, shape, contour, location, sound, timbre, color, temperature,filter, vibration, tone, effect, texture, luminosity, electromagneticradiation spectra, modulation or other emission or other aspect of thatpart), component, attribute, characteristic, aspect, filter or othereffect of, or other thing substantially impacting or affecting, an imagethat remains constant or at least relatively constant or constant insome respect within a parameter over the course of a function of theimage or image stream or from image-to-image in a series of images (orpart of such a series) of which the image is a part, or that remainsconstant or at least relatively constant or constant in some respect asdata changes in an associated database. In addition to RepresentationalFixed Elements, Fixed Elements may include Non-RepresentationalElements, such as metadata and Relational Aspects, as further defined inthis application. Image Elements include their alternate storage andtransmission forms, for example as encoded signals and/or other storageor transmission media and formats, that may require furthermanipulation, translation, rendering, projection or other treatment tobe experienced by a user as a part of an image, as further defined inthis application.“Fixed Representational Element” means a Fixed Element, as furtherdefined in this application, which relates to or affects therepresentation or representational nature of an image or understandingor cognition by one experiencing the image or representation of theimage.“Semi-Dynamic Element” means a part (including, but not limited to, anysize, parameter, shape, contour, location, sound, timbre, color,temperature, filter, vibration, tone, effect, texture, luminosity,electromagnetic radiation spectra, modulation or other emission or otheraspect of that part), component, attribute, characteristic, aspect,filter or other effect of, or other thing substantially impacting, animage that may change in some way or respect over the course of afunction of the image or image stream or from image-to-image in a seriesof images (or part of such a series) of which the image is a part, orthat may change in some way or respect as a result of data changing inan associated database, but, that may also remain constant or at leastrelatively constant or constant in some respect within a parameter overthe course of a function of the image or image stream or fromimage-to-image in a series of images (or part of such a series) of whichthe image is a part, or that may remain constant or at least relativelyconstant or constant in some respect as data changes in an associateddatabase. In addition to Representational Semi-dynamic Elements,Semi-dynamic Elements may include Non-Representational Elements, such asmetadata and Relational Aspects, as further defined in this application.Image Elements include their alternate storage and transmission forms,for example as encoded signals and/or other storage or transmissionmedia and formats, that may require further manipulation, translation,rendering, projection or other treatment to be experienced by a user asa part of an image, as further defined in this application.“Semi-Dynamic Representational Element” means a Semi-dynamic Element, asfurther defined in this application, which relates to or affects therepresentation or representational nature of an image or understandingor cognition by one experiencing the image or representation of theimage.“Relational Aspect” means some relationship which may be sensed,understood, recorded and/or implemented between an image or DynamicElement, Fixed Element or Semi-Dynamic Element thereof and another suchimage or element or external factor, for examples, a 3-dimensionalsubject similar to that represented in an image, which may be2-dimensional, or a change in news relevant to the content of a report.Relational aspects may be multi-variate, with complex applicationfunctions and all image elements may be set to have multiple RelationalAspects with every other image element.“Interactive Element” “Interactive Aspect” or “Interactive Parameter,”in addition to the senses of their individual words, each mean a part(including, but not limited to, any size, parameter, shape, contour,location, sound, timbre, color, temperature, filter, vibration, tone,effect, texture, luminosity, electromagnetic radiation spectra,modulation or other emission or other aspect of that part), component,attribute, characteristic, aspect, filter or other effect of, or otherthing substantially impacting or affecting, an image that may beaffected, changed, added to or taken away from or, at least in part, theresult of at least one Relational Aspect.“Variation Toleration” means a degree, amount or pattern of change, or adegree, amount or pattern of change of such change, in some part(including, but not limited to, any size, parameter, shape, contour,location, sound, timbre, color, temperature, filter, vibration, tone,effect, texture, luminosity, electromagnetic radiation spectra,modulation or other emission or other aspect of that part), component,attribute, characteristic, aspect, filter or other effect of, or otherthing substantially impacting, an image or external factor that, ifsensed or otherwise understood or established as existing, will notinterfere with the system determining or defining an Element orRepresentational Element, including defining whether such an Element orRepresentational element is Dynamic or Fixed, assigning a uniqueidentifier to the element and determining whether to exert otherelements or treatments or affects upon such element.“Element Seeding or Borrowing” refers to a system according to aspectsof the present invention taking and applying elements, which may bedynamic and from a variable or even live source and infinitely sodynamic and variable, and which may be previously unrelated to thesystem and may be an ambient source. For example, rather than store andrecall complete dynamic texture information for a Fixed Element, atexture template tag may partially reflect actual ambient texture, whichmay or may not ever be digitally or otherwise stored separately withinthe system.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 depicts example representational elements of an image from aseries of related images, as might be treated or generated by aspects ofthe present invention.

FIG. 2 is a graphical depiction of fixed and dynamic representationalelement identifiers of the image series, FIG. 1 representing one imagein such a series, which identifiers might be assigned and used forrecording and exerting fixed and dynamic elements in aspects of thepresent invention.

FIG. 3 depicts another example of representational elements of anotherimage in the same image series as involved in FIGS. 1 and 2.

FIG. 4 depicts an example database associated with image elements inaccordance with aspects of the present invention, corresponding with theimage series of FIGS. 1-3.

FIG. 5 depicts an example image series for a coloration and/ortexturization representational source image that may be overlaid onto aglobe background representational image element in accordance withaspects of the present invention, as shown in FIGS. 1 and 3.

FIG. 6 depicts an example background representational image elementsource, for the “globe with a stand” Fixed Representational Elementshown in FIGS. 1 and 3.

FIG. 7 is a perspective drawing of an exemplary image capture activity,as might occur in using aspects of the present invention, involving amovie camera capturing a scene with a car colliding into a fire hydrantduring a sunset.

FIG. 8 is a graphical depiction of Fixed Elements, Dynamic Elements andSemi-Dynamic Elements, as might be defined by aspects of the presentinvention upon digesting the capture activity depicted in FIG. 6.

FIG. 9 is a graphical depiction of an example resulting image as mightbe rendered in accordance with aspects of the present invention, fromthe capture activity depicted in FIG. 7.

FIG. 10 depicts an exemplary element seeding or borrowing source, andcapture activity involving the same, as might be used in accordance withaspects of the present invention—specifically, a seed dynamic texturesource, as might be used in the image capture and generation depicted inFIGS. 7-9.

FIG. 11 is a depiction of part of a user interface that might be used bya user implementing certain aspects of the present invention.

FIG. 12 is a block diagram of some elements of a system in accordancewith aspects of the present invention.

FIG. 13 is a process flow diagram depicting exemplary steps that may becarried out by a control system, or combination of control systems,implementing exemplary programming, methodology and other aspects of thepresent invention.

FIG. 14 depicts an image capturing activity, comprising a camera'sviewfinder, framing a scene that includes a dynamic image element, andsome exemplary forces and resulting movements that may be captured andrendered by a system and method managing images in accordance withaspects of the present invention.

FIG. 15 depicts the an image capturing activity, comprising the samecamera viewfinder framing a scene similar to that pictured in FIG. 14,and a force vector resulting from a new physical model selected andapplied by a system capturing and rendering dynamic elements for imagemanagement in accordance with aspects of the present invention.

FIG. 16 is a schematic outline of several components of a new image fileformat, with embedded physical model data for managing virtual objectsas subjects of corresponding depicted dynamic image elements.

DETAILED DESCRIPTION OF THE DRAWINGS

FIG. 1 depicts example representational elements of an image from aseries of related images, as might be treated or generated by aspects ofthe present invention. The representational elements of FIG. 1 show areport regarding human demographics of a continent of the Earth. Imageelement 101 is such a representational element, and represents andconveys to a viewer the impression of a globe depicting the Earth, witha representational element of a bracket stand 103, and coloration ortexturization representational element 105 (specifically, depicting andhighlighting the continent of South America). As will be explained ingreater detail below, coloration or texturization representationalelement 105 impacts image element 101, according to certain aspects ofthe present invention. Heading text representational element 107explains the overall subject of the image to a viewer. Pie chartrepresentational element 108 represents and conveys to a viewer theimpression of a pie chart showing demographic data. Another pie chartrepresentational element 110 represents and conveys to a viewer theimpression of a chart similar to 108, but showing demographic data fromanother year. Pie chart year text representational elements 112 aresituated near pie chart representational elements 108-110, explaining tothe viewer which year the data of each pie chart element demonstrates. Abackground representational element 114 provides to a viewer the conceptthat the entire image relates to the continent of South America, bydepicting a rough outline of the shape of the continent of SouthAmerica. Due to aspects of the invention that will be explained ingreater detail below, background representational element 114repositions or otherwise affects pie chart elements 108 and 110 byshowing through it, at least to some degree, changing some of therepresentational aspects of element 114 through Relational Aspectsettings or other settings set by the system or user.

Bar infographic heading text representational element 116 explains thesignificance of certain elements that appear below it, namely horizontalbar representational elements 118-122, which depict to a viewer a visualdemonstration of the relative amount and change in life expectancy inSouth America in certain years. Textual year indicator representationalelements 124-128 set forth those years for a viewer. Life expectancy inyears textual representational elements 130-134 set forth for the viewerthe number of years corresponding with horizontal bar representationalelements 118-122. Specifically, life expectancy in years textualrepresentational element 130 sets forth for the viewer the number ofyears corresponding with horizontal bar representational element 118,while textual representational elements 132 and 134 correspond withhorizontal bar representational elements 120 and 122, respectively. Aswill be explained in greater detail, below, horizontal barrepresentational elements 118-122 impacted life expectancy in yearstextual representational elements 130-134 by determining their finalposition in the image.

FIG. 2 is a graphical depiction of potential aspects of a user interfacethat might be used in an embodiment of the present invention, and thatmight be so used to generate the image series related to FIGS. 1 and 3.More specifically, FIG. 2 depicts fixed and dynamic representationalelement identifier tags of the image series, placed in an exemplaryCartesian coordinate space by a user, FIG. 1 having represented one suchimage in such a series. These element identifier tags might be assignedand used for recording, manipulating and exerting (or causing to beincluded in an image, set of images or image stream) Fixed Elements,Dynamic Elements and Semi-dynamic Elements, among other things, inaccordance with aspects of the present invention. In certain aspects ofthe invention, a computer system includes computer hardware and isconditioned with specialized software creating a user interface thatallows a user to create, layout, incorporate, assign or associateidentifier tags at coordinates in a 2-D, 3-D or otherwisemultidimensional representative Cartesian plane or space, or similarrepresentative area or space, such areas and spaces being known in theart of creating general user interfaces. (Aspects of the preferred userinterface to implement aspects of the present invention and an overviewof a potential system are described in greater detail below, withrespect to FIG. 11, among other places.) In accordance with aspects ofthe present invention, an identifier tag allows a user to indicatepositions on a Cartesian plane or space on which Fixed RepresentationalElements, Dynamic Representational Elements and Semi-DynamicRepresentational Elements will be associated, and also allows the userto create or associate other image elements, database elements, metadatacomponents, Relational Aspects, Interactive Aspects or elements forElement Seeding or Borrowing, which may be previously unrelated to thesystem. A user may also generally or otherwise associate variouselements, with no Cartesian coordinate association, which may otherwisealter or impact an image, set of images or image stream generated by thesystem. For example, the user may change the file format(s) in which agenerated image, set of images or image stream is saved, some of whichmay have no background color filling in gaps (as is possible with .pngformat files) or may provide for automatic posting in viewable locationswith filters and other workflow executed in a predetermined sequence onthe files or in creating the elements of the files beforehand, through aworkflow template (e.g., scan or gather designated source, buildmetadata, save as .png). In any event, the user may also use the userinterface to cause the generation of an image or set of images or imagestream after performing the creation and layout of identifier tags andotherwise incorporating, assigning or associating elements, settingRelational Aspects and Interactive Parameters and may also dictate otheraspects of such image, set of images or image stream generation, asexplained in greater detail with respect to FIG. 11, and elsewhere,below. According to aspects of the present invention, a set of suchidentifier tags, arranged in a Cartesian coordinate system with the usersettings discussed above, which may be enriched by other associatedtags, data, relational data and other aspects that may be set by theuser (such as workflow templates), may be “saved” for later use, afterwhich time it may be “reloaded,” further altered and saved again,optionally, as a new version. These saved composites may be termed“templates,” as used in present invention, and may include multiplepages or frames of such templates, which may be further organized intotiers and super-tiers, and also arranged or ordered for generationsequences, which may depend on user input (e.g., video game playercommands through a gaming interface, viewing the output from thesystem). For example, a user might use aspects of the present inventionto create, layout, incorporate, assign or associate the followingidentifier tags into a one-page template, which may be saved for lateruse:

Heading text identifier tag 201 is of a Fixed Representational Elementnature (as may be selected, and later changed, deleted or modified bythe user), describing text that may be placed by a user to create adescriptive heading for an image, set of images or image stream to begenerated by certain aspects of the present invention. If the user wereto generate an image, set of images or stream of images from a templatewith aspects described in FIG. 2, the result would include the placementof the same text in each generated image or point in the image stream,assuming that no further time or function condition or limitation to itsdisplay is applied (though such functions are possible, as explainedbelow). The user may also set a number of other parameters, RelationalAspects and associated data, such as Seed or Borrowed Data to generate avisible texture, hyperlinks, font selection and library, background orforeground effects, source subject information and element interactioncharacteristics, including, but not limited to, a “z-index” parameterfor indicating whether other elements would lie below or above theresulting generated Fixed Representational Element, based on its ownz-index, if the two elements were to overlap by an arrangement laterchosen by the user. Continent heading text identifier tag 203 is of aDynamic Representational Element nature, describing text that may beplaced as part of a descriptive heading for a particular resultingimage, image within a set of images, or point in an imagestream—specifically, the particular continent to which the image orpoint in an image stream pertains. If the user were to generate animage, set of images or stream of images from a template with aspectsdescribed in FIG. 2, the result would include the placement of text andother aspects dictated by the system that vary depending on the image,which variation may be dictated by an aspect of an associated database.For example, a database associated with the template correlating withFIG. 2 might be in the form of a .csv file, Excel file or MySQLdatabase, with rows (the horizontal arrangement of cells structure) andcolumns (vertical arrangement of cells structure) of data, and each rowmight define aspects of each image generated by the system in a seriesor some amount of a stream of images (which amounts might overlap) whilecolumns dictate which identifier tag is to receive, generate or beimpacted by the data in the particular column. The database can be ofany other type and configuration known in the art that might providedata on an image-by-image, or point in a function of an image stream,basis. Further aspects of such a database are described in FIG. 4,below. As with tag 201, the user may also set a number of otherparameters (such as Interactive Parameters), Seed or Borrowed Data,Relational Aspects and associated data with respect to tag 203. Asanother example, and with particular application to tag 203, interactiveparameters may be set such that a side or corner of the tag and itselement output remains fixed at a point while, if the particular size ofthe generated representational element varies, the change in spaceoccupation occurs at the location of the opposing sides and corner ofthe tag. Such a parameter could be directly determined by a user, orautomatically determined according to other selections by the user—forexample, if the user selects to “left justify” the resulting text, alower left corner fixed point could be determined automatically. Asanother example, interactive parameters may be set to control theeffects on and from other elements, such as other representationalelements set by other identifier tags. On that score, turning to headingtext identifier tag 205, which is also of a Dynamic RepresentationalElement nature, it may be possible for the user to set parameters of tag203 that affect tag 205 and its element output. For example, the usermay set the two tags to collide, giving a “pinning preference” (or “pinpreference”) to tag 203 and tag 205 such that, if the two resultingelements were to overlap when generated from the tags, the elementresulting from tag 205 would “bounce off” of (in other words, notoverlap in occupied area and otherwise react according to a physicalmodel) or deform in response to the element generated by tag 203. Ifboth tags are given an equal pin preference, they might both sharedeformation and bounce effects in their resulting elements or,optionally, proportionally according to size, mass or othercharacteristics of the tags or resulting generated elements set by aphysical model and/or physics engine (hereinafter, collectively,“physical model”) of their behavior. As another example of interactiveparameters, in other instances, if adjacent or otherwise associated tagsprovide for different clauses of a sentence, variations of phrasing(such as pluralizing or singularizing nouns and other grammatical orother logic) might automatically change aspects of one or each resultingelement in reaction to the other. As another example, the tags mightcoordinate color schemes or vertical positioning, or select otherreacting elements (for example, by system logic), to match the patternbetween elements. As another example of an interactive parameter oraspect, any element that tag 205 generates may be caused to “stick” ormove along or against the edge of the corresponding element generated byadjacent tag 203 in particular directions and orientations (includingchanges thereto over a time or another function) or they may apply forcevectors in particular directions against one another, as determined bythe system and user. In this way, any gap or incongruity that mightotherwise result from the implementation of Dynamic RepresentationalElements can be cured, and other desirable dynamic effects may beimplemented. The user may access any tag to both define and create newinteractive parameters, or associate new image elements, databaseelements, Interactive Aspects, metadata components, Relational Aspectsor elements for Element Seeding or Borrowing (replacing or in additionto those existing and already associated).

Globe background identifier tag 207 is of a Fixed RepresentationalElement nature, and would cause the system to generate a background“globe with stand” image element if the system were commanded togenerate an image, set of images or image stream, in accordance withaspects of the present invention. Such a background image might beaccessed from a stored source (“library”), as selected in a userinterface, according to aspects of the present invention and, forexample, might be the background image depicted in FIG. 6. Turning ourattention back to FIG. 2, globe foreground identifier tag 209 is of aDynamic Representational Element nature, and, based on its positioningover globe background identifier tag 207 and, optionally, upon analignment linking aspect, whereby the program determines a commoncoordinate system for tags 207 and 209 and ensures that they are alignedproperly, and with a properly populated and assigned database, the usermay dictate that tag 209 cause the correct continent feature of theresulting globe with stand image to be colored by a world continentoverlay element which may be from an image library that may be assignedusing tag 209 with the user interface. Thus, if the user were togenerate an image, set of images or stream of images from a templatewith aspects described in FIG. 2, the result would include the placementof the overlay element associated with the appropriate continent towhich the particular image pertains. This may be accomplished, in part,by a database with rows and columns, as discussed above, where onecolumn of data contains linking information that is dynamically matchedwith the appropriate overlay element from an image library, as in, forexample, FIG. 5. Thus, the user may use tags to indicate which column ofdata should be dynamically matched with an image library at a tagposition, in addition to dictating whether to so match data with animage library, or print data from the database in a particular column,itself, at the tag location, or both. Optionally, the same effect may beaccomplished by the user using the user interface to create onesemi-dynamic representational element tag, with the abilities andfeatures of both the background and overlay tag, 207 and 209. In eithercase, as will be explained in greater detail below, the user may alsocontrol the size, proportions, centering or justification, opacity,fill, blending mode, orientation and other characteristics of thedynamically matched image-based image elements, in addition to otheraspects discussed above with respect to textual elements, as well asdictating the location and characteristics of text, if also printed, inrelation to the matched image-based image element, and vice versa. Forexample, the user may dictate that the text element appear to “stick” ormove along, against or near the edge of the corresponding image-basedimage element generated, in particular directions and orientations(including changes thereto over time or another function), regardless ofthe size of the actually matched image-based element in a particularimage, image series or stream of images at a particular point along afunction of that stream. The user may also dictate the distance at whichthe text and image-based representational element situate near oneanother, and on what side or at what point or degree of orientation theyso situate. To match data with images, it is preferred that the data inthe column chosen in the user interface for the tag in question containat least a part of the actual name of the image-based representationalelement in the appropriate library, and that a default image library (oran image library selected by a function or variable function) beautomatically selected that corresponds with the appropriate column ofthe database, but, alternatively, the user may dictate or change theimage library, and may gain more control over template design that wayby selecting more image libraries that may be of his or her own design.

Bar graphic identifier tag 210, in conjunction with some of itspotential element output for different images in an image series ordifferent points in an image stream as shown in other figures,illustrates an example where a tag is used to dictate the inclusion ofboth an image-based element and the data itself, and to further mandatethat the data appear to stick or otherwise orient to the right-hand sideof the image-based element, thus producing a compound element. Compoundelements may be subject to additional control parameters that allow thesystem or user to dictate order, apparent mounting position between thetwo elements within an image, degree of margin between the two elements,and direction of alignment and arrangement between the two elements.

As with tag 207, continent background tag 211 dictates a backgroundrepresentational element but, in this instance, a dynamic one.Specifically, under aspects of the present invention, the system wouldcause a large background element—assuming that resizing element outputto match tag size is selected as a parameter (which tag size maynonetheless constrain original image proportions, if so selected by theuser) and assuming a supporting library that can be matched to databasecolumn data (or other source) is included or linked to the system at theselection of the user—to be created. In the instance of backgroundelement identifier tag 211, and as shown in its element output, examplesof which are depicted as 114 and 314 (a part of FIG. 3, which will bediscussed below), the user may have selected a warping RelationalAspect, which causes an impact on foreground representational elementspotentially generated by tags 213, 215, 217 and 219, as will beexplained in greater detail below and in relation to FIGS. 1 and 3.Identifier tags may place image elements at an original size in theCartesian plane of the user interface, or the user may resize them, orotherwise change their appearance, with parameters of the identifiertags and the user interface in general, discussed in greater detailbelow, for example, in relation to FIG. 11, discussing aspects of apotential user interface.

FIG. 3 depicts a second example of representational elements of anotherimage in the same image series (or another point in an image stream)potentially generated by aspects of the present invention as involved inFIGS. 1 and 2. By comparing differences and similarities between theresulting representational elements of FIG. 1 and FIG. 3, which aredepictions of two different images in an image series, or two differentpoints in an image stream, certain aspects of the present invention canbe understood. As with image element 101, image element 301 is arepresentational element and represents and conveys to a viewer theimpression of a globe depicting the Earth, with a representationalelement of a bracket stand 303, and coloration and texturizationrepresentational element 305 (specifically, depicting North America),comparable to elements 103 and 105 of FIG. 1, respectively. The imagesources and potentially resulting representational element 303 is notsubstantially different from element 103. However, coloration ortexturization representational element 305 affects a different location(the continent of North America) than representational element 105 fromFIG. 1 (the continent of South America). In addition, warping andtexture or coloration matching and overlay parameters set by a userinterface and/or by the system have led the resulting elements of 105and 305 to be oriented in space differently from one another—eachcorrectly to match the borders of the outlined continents on which theybelong in element 103 and 303, respectively, and with perspective and3-D source data (a shere) dictating their final orientation and adjustedappearance for element output by the system. This dynamic difference,and others affecting image elements, can also result, according to thepresent invention, from differences in: associated data, for instance,in parts of a database or source and/or image library or other imagesource, other parameters, Relational Aspects, Element Seeding orBorrowing, the source seeded or borrowed, which may correspond with theimage within a set of images or point in an image stream. For example,if a row of a database accessed by the system indicated in a column ofdata selected by the user for identifier tag 209 that the system shouldaccess a .png image named “North America.png” from an image librarycorresponding with that column, and if identifier tag were properlyaligned, in a higher z-index and above identifier tag 207 (for the globebackground) the system would place that image above a background imagefor the globe, thereby coloring, highlighting or texturizing the portionof the globe image that is the subject of this report (depending on theopacity of the image and other parameters for texturization effects,such as blending modes, selected by the user) for FIG. 3. According tothe same process, the system would access a different row of data,associated with South America, in creating the image depicted in FIG. 1,and accessing another image from the same image library selected byidentifier tag 209. Other techniques of originating, controlling,tweaking and resetting such dynamic effects are also within the scope ofthis invention. For instance, among the many parameters that a usermight set, a user might indicate by using an interface that element 305be partially transparent, rather than completely opaque, or that avariety of image blending aspects occur (such as overlay, darkening,lightening, multiplying, dividing and many other known, related blendingmodes) with the result of preserving or blending aspects of theunderlying fixed globe image to some degree where there is overlap. Itis also within the scope of the invention that streaming data or ElementSeeding or Borrowing, including fixed and dynamic textures andcolorations, be used to create such dynamic effects, rather than adatabase, while still yielding such a dynamic change from point to pointalong an image stream function. It is also within the scope of theinvention that a single semi-dynamic identifier tag be used, rather thanmultiple identifiers tags, thereby conflating the features of bothidentifier tags 207 and 209, and thereby potentially easing the properplacement and interrelationship between the two resulting elements.

Similarly, different associated source data, parameters, RelationalAspects, Element Seeding or Borrowing, seed or borrowed source (each ofwhich may correspond with the image within a set of images, or point inan image stream) may lead to differences between header representationalelement 107, in FIG. 1, and 307, in FIG. 3. Heading identifier tag 201,as potentially created and placed by a user as a fixed element, createsthe same textual element part (namely, the first four words of theheading) in both element 107 and 307. However, the dynamic elementidentifier tags 203 and 205 yield different results in the image createdthat is depicted in FIG. 3 when compared to its counterpart in FIG. 1,owing to different associated dynamic data or Element Seeding orBorrowing. Certain interactive parameters, which may be variably set bythe user, explain the seamless presentation of both dynamic elements setby identifier tags 203 and 205, even though the resultingrepresentational element components are of a different length than thoseof FIG. 1, namely “Continental North America” and “Jun. 5, 2012,” forFIG. 3, versus “South America” and “Aug. 5, 2012,” for FIG. 1. Namely,the user may have set a collision interactive parameter between tags201, 203 and 205, with a left-side pin and pinning preference to tag201, 203 and 205, in that order of priority, in conjunction with a“shrink-to-fit” parameter (although other parameters, such as awrap-to-next-line parameter, may also be set, based on the workspace,leading to overflowing text flowing to the following line space of thereport) leading to the line of text shrinking or reducing in font orother size-bearing characteristics in FIG. 3, in comparison to FIG. 1.Any similar parameter may be automatically set as a function of using atext-creating tool in creating the identifier tags. Likewise, differentassociated source data, parameters, Relational Aspects, and ElementSeeding or Borrowing may explain differences between the set of piechart representational elements in FIG. 1 and FIG. 3, namely,representational elements 108-110 and elements 308-310, respectively.For example, different source data associated with the image depicted inFIG. 3 could be matched by the system to different identified imagesfrom a pie chart image library, and then incorporated in the image, suchas data of “70%” being matched by the system with a 70.png image fromthe library associated by the user in identifier tag 215, using a userinterface (by an indexing and title matching function of an automaticquery and producing resulting found image based on data in cell searchengine, for example). Interactive parameters could also explain theresulting seamless and effortless resulting interaction between element308, and the background element 314, which element 308 now overlaps thebackground element 314 of a lower z-index, but allowing some data fromthe lower image to pass through, in FIG. 3—potentially, as a translucenteffect, or overlay parameter. The usefulness of a compound element tagis shown in representational elements 318, 320, 322 and 330, 332 and334. For example, by setting both a dynamically matched image elementand a text element in one element, such as with identifier tag 210, andthen setting a parameter for the text element to mount directly, at 90degrees, to the right-hand side of the image element, with text notwarped or rotated, with a few millimeter margin, the resulting numericaldata is always clearly presented alongside the resulting bar of the barchart, which may vary greatly from image-to-image in a series, or frompoint-to-point in an image stream, generated by the system. It is withinthe scope of this invention that many elements (far more than 2), ofmultiple types, may be associated in such a compound tag, withseparately-set parameters, such as mounting parameters and physicalmodel parameters for originating 3-D or 2-D objects (or “subjects” ofthe image elements), which models may include other aspects andelements, many of which may not be representational.

FIG. 4 depicts an example database that might be used by a systemaccording to aspects of the present invention to create the images orpoints in an image stream depicted in FIGS. 1 and 3, from aspects of auser interface depicted in FIG. 2 (and FIG. 11, which will be discussedseparately, below). The database depicted in FIG. 4 contains a columnand row structure that is commonly used in certain database systems,such as Microsoft's “Excel” software, Oracle's Open Office “Calc”program and MySQL, to name just a few with this type of structure. Theleading or top row 401 may be used by the user and system as a “title”row, meaning that the data contained in its cells, 403-415, serves toprovide titles that aid the user and system in recognizing data in therows below each of them. For example, the title in cell 405, “ReportDate”, aids the user and system in recognizing that an identifier tagcontaining the same characters, “Report Date,” (potentially with codingtags to key that a database call and response is required, such as theleading and trailing % symbols shown) requires that data from the columnbelow that heading be placed at the coordinates or according to theother parameters of such an identifier tag into a resulting image, setof images or image stream. The rows of information other than the titlerow, which are depicted as numbers 417, 437, 455 and 457, each containthe data for each heading and column that corresponds with one image orpoint or range in an image stream to be output, rendered and/orgenerated by the system. In the instance of an image stream with ranges,cells might set out time signatures or other indicators of duration ortype of presence for an element, or such parameters might be set outseparately from another data or streaming source. Such database sourcesmight be modular and freely arranged and associated in the sense thatthey need not function row-by-row in order, but may instead have pointsof data or streaming sources that overlap in time encoding or signaturesand other functions and may have multiple interactive properties orlogical controls, interrelating with, and depending on, other sourcedata and other conditions external to the database and/or sensed orinput by the user or system, causing some data to be processed in anorder other than top-to-bottom to generate images, and in sequences thatdiffer column-to-column. Nonetheless, in the present example of apreferred embodiment of aspects of the present invention, one rowgenerates data for each image of an image set or point or range in animage stream. It is preferred that other parameters controlling aspectssuch as positioning, interactive features and logic underlying thedisplay and imaging features be controlled in a separate database orstorage medium that may not be visible to all users, thereby making theuser interface more user-friendly, and making the column and rowdatabase have one clear function to a user accessing it—namely todictate the content, order and/or general duration of dynamicrepresentational elements in an image, image set or image stream. It iswithin the scope of the invention that the data and identifier tags berepresented differently, in a user interface with an image-by-imagepreview function and other controls for more easily changing the data.For instance, a user might simply click on, click and hold and move thecursor to resize or dial back a bar graph or pie chart component, orother such element, in the preview and adjusting positioning,orientation and sizes of such previewed elements, which leads the systemto cause a change in the underlying data in the database for thecorresponding row of data to match the new image element selected.

FIG. 5 depicts an example image series for a coloration andtexturization representational element that may be overlaid onto abackground globe image element, as shown in FIGS. 1 and 3, according toaspects of the present invention. As explained with respect to FIGS. 1,2 and 3, a user interface may allow the creation of a single identifiertag, and may create a single semi-dynamic element with both a backgroundimage and the coloration element as components incorporated. However, itis also possible to have separate identifier tags, that nonethelessbecome linked or otherwise associated with Interactive Aspects byparameters set by the user, yielding substantially the same result, orhaving two images placed in conjunction—one for the fixed background ofthe globe and stand (a Fixed Element) and one of the appropriate overlayelement for the particular image (a Dynamic Element, changing with eachimage, image in a series of images, or point or range in an imagestream). As explained with respect to FIG. 4, a data source may accessand link the appropriate image library 501 for the continent overlayelement, (as such a library may be variably created and selected by auser in conjunction with an identifier tag, such as identifier tag 209)and a system according to aspects of the present invention will accessthe image from the library that matches identifying information from adata-source that is identified with the identifier tag, placing thecorrect continent overlay element relating to the image to be created,onto the fixed background image and, optionally, applying otherRelational Aspects and parameters (such as orienting and warping to“best match” underlying image boundary patterns) and merging andflattening them into one image representational element. It is withinthe scope of this invention that image libraries themselves be dynamic,incorporating additional data and sources, or that an external database(such as one on the internet) be accessed and searched and that matchingbe evaluated based on any search criteria variably set by the system oruser, not just image titles matching data in a data source.

FIG. 6 depicts an example background image, 603, for the “globe with astand” Fixed Representational Element shown in FIGS. 1 and 3, as it mayappear in an operating system explorer view, with its title (in thisinstance, “GlobeBackground.png”) below its display. It is this imageelement that may be located in an image library, as shown in displaywindow 601, just as with the dynamic image library of FIG. 5, to formthe background for those overlay elements depicted in FIG. 5.(Incidentally, the same type of window view may be used to experiencethe library discussed in FIG. 5.) With the exception of properties,parameters and aspects relating to dynamic data matching from adata-source, all of the same abilities of the identifier tags and userinterface discussed above with respect to FIGS. 1-5 may apply to theidentifier tag (207) yielding the fixed background element thatapproximates or is the appearance of image 603 within resultinggenerated images. Identifier tag 207 allows a user to select abackground image library, 605, and further select a particular image,such as 603, that will be implanted as a representational element ofevery image, set of images or stream of images set forth in an imagegeneration run, which may be initiated, maintained and directed todestination location(s) by the user, for example, by internet orintranet sharing, e-mail, instant messaging, conventional printing,display, performance and broadcasting, or any other known means ofdelivery for images. The fact that the identifier tag is fixed, and thatthe background image itself may be of a fixed form (although it may alsobe non-fixed, as with .gifs and updating images from another source)does not mean that the representational element as is incorporated intoan image must be completely fixed in nature. As explained previously,for example, Relational Aspects, parameters and Element Seeding orBorrowing may greatly change the appearance of such a fixedrepresentational element, and may do so according to a function orstream, which may vary from image to image, for example.

FIG. 7 is a perspective drawing of an exemplary image capture activity,as might occur in implementing apsects of the present invention,involving a movie camera capturing a scene with a car colliding into afire hydrant during a sunset. Generally speaking, image capture (as asingle image, set of images, image stream, or image elements andcharacteristics) aspects of the present invention can be thought of asseparate or overlapping or complimentary to the image creation andgeneration aspects discussed above, and may be used in conjunction withthem for a seamless image capture, processing and distribution work-flowthat can be extremely efficient and yield greater realism andverisimilitude, and with a greater, more intense and more helpful orenjoyable impact on a user experiencing the results of the invention.The image capture aspects of the present invention include the abilityto interpret, record and re-represent sensed images as similar orenhanced composites of fixed, dynamic and semi-dynamic representationalelements, or as data and tags that would, under the operation of thesystem, generate such similar or enhanced composites of fixed, dynamicand semi-dynamic representational elements. Camera with tripod 701 isset up to capture a scene taking place before it. 701 may include anysensory and sensory transmission or conveying apparatus, such as a stillcamera, a moving picture camera, microphone or a hybrid sensory device,such as, but by not limited to, a digital single lens reflex camera(“SLR”) with both separate picture and movie capturing capabilities, anda built-in microphone, such as the Canon EOS 5D Mark II. 701 may also bea more advanced, direct conveyance or all analog or optical formattransmission medium, not requiring a digital sensor or storage medium,but still allowing the transfer of streaming media, such as by fiberoptic cables. Many other methods of capturing images are known in theart, and within the scope of this invention. In this example, camerawith tripod 701 is sensing and storing or transmitting a scene, ortransmission medium encoding the same, including a sky 703, sun 705,horizon 706 (into which the sun 705, if a real or accuratelyrepresentative theatrical set piece sun, may be setting), a relativelyflat ground landscape 707, and a moving vehicle 711 driving partially onand partially off of a roadway 709 that collides with a fire hydrant713, which becomes partially knocked over and ruptured, causing atorrent of water 708 to rush skyward like a geyser.

In FIG. 8, one can see a depiction of how this same scene might becaptured, tagged and summoned forth for image creation and distributionby a system in accordance with aspects of the present invention. Forconvenience, the same basic perspective is given as that provided inFIG. 7, although it is understood that the camera would have to be movedto gain the perspective of the figure. The system first may sense andrecognize, based on image capture activity, which may include acomparison of different frames and/or data from more closely relatedtime periods, that at least a majority of the sky region 703 isrelatively unchanged from moment-to-moment, and create an identifier tag803. Although the coloration, size, movement (shifting) and shading of703 may change subtly from moment to moment, due to the setting of thesun 705 as a light source and other atmospheric factors, the system mayemploy Variation Toleration (which may be adjustable by the system oruser by an amount of toleration control, and so adjustable on anitem-by-item or functional basis) to avoid identifying the region fordynamic representational element treatment, and thereby avoiding savingor transmitting specific data covering each slightly changed aspect thatis below threshold of the Variation Toleration, such as a slight pixelchange, as an image file, part of an image file, series of image filesor image stream. In addition, the system may sense boundary lines,similarity in multiple images of an element that has moved, in terms ofsize, shading, lightness, color or texture, or 3-D subject matching, incaptured images to describe or supplement image boundaries betweenelements. In generation, or after distribution, however, realism,including the vibrance of very slight but realistic changes in such“Fixed Representational Elements” (or even other, Dynamic Elements) maybe added in by a function approximating similar observed slight changes,which may include Element Seeding or Borrowing from other sources, suchas from a library of luminance, coloration and scintillation effects,functions, integrals, derivatives or other formulas for similar slightchanges. Upon recognizing and capturing regions as relatively fixedrepresentational element tags, a multivariate search tag encoding can berecognized, assigned and/or recorded to ease such a search and matching.For example, a code such as “BHSNMCLRMLFG” might be recognized orencoded by the system, where the recognition aspects or encoding standsfor “Blue Hour Sunset New Mexico Clear (sky) Medium Light Foreground”and is so understood by the search system aspect, which uses appropriatesearch indexing and library structuring to facilitate later retrieval ofsuch dynamic elements, elements, aspects, Element Seeding or Borrowingand effects to alter or condition the fixed element potentiallyresulting from identifier tag 803. The same potential aspects may beapplied to identifier tag 805, which is created in reaction toperceiving, channeling or conveying the setting sun, 705. While someaspects of the sun will change slightly, the system may be programmed totolerate such changes, while recording or associating useful informationfor reconstructing very similar or even more richly enhanced, striking,useful or compelling changes when generating an image, set of images, orimage stream from the image capture activity. In addition, we see a newtype of identifier tag feature, whereby an outline preview of thepotentially resulting element can be seen, as the outline of theboundaries of the sun within or about the identifier tag 805. Similaraspects of the invention may apply to the relatively unchangingbackground elements 707 and 709, and the identifier tags assigned by thesystem in a Cartesian plane or virtual space user space, namely, tags807 and 809. It should be noted that, with respect to camera shift, yaw,pitch, descent, ascent or vibration, that the Element Seeding orBorrowing can be from a relatively few still shots of the scene, thatare encoded for a shifting algorithm or function by the system, and onlythe new or dynamic aspects of the shifted scene need be newly recordedfrom image to image or point to point in the image stream. Furthermore,because more dynamic subjects, tags and elements tend to be centered orcontrolled in predictable points as subjects of an image, series orimages or image stream, such as at the points according to the “rule ofthirds,” the system may easily reposition such dynamic elements andtheir identifier tags in such locations, and filling in regions at theperiphery of the frame from data from Fixed Element sources or othersampling taken before moving of the camera, thereby reducing oreliminating “camera shake,” hand tremor and other aspects of thecomposition of a series of images or of an image stream that reducefluid, steady viewing.

Turning next to the tag created to identify certain aspects relating tothe moving car, 811, a system according to the present invention maysense boundary lines or movement, including the movement of shadows andreflections, especially when comparing neighboring and other images inan image series, or different points in an image stream, to define aregion in the user space (such as 2-D Cartesian, or 3-D virtual space)and image(s) or image stream to be produced, identified with the movingcar and to store image, image change, movement functions and otherequations, Relational Aspects and other data defining a representationalelement for the car, including, with logic to fill in gaps and, asnecessary, from information from multiple views of the car, to define itas a 3-D subject giving rise to the car representational element. 3-Dphysical model characteristics (such as a mesh or filled, textured andrendered 3-D object) can be captured and incorporated by observing thesource object subject movement and deformation. All of this informationmay be associated and accessible for manipulation in identifier tag 811,created by the system.

The system is not restricted to assigning one identifier tag per elementwith perceptible 2-D or 3-D boundaries or other characteristics.Multiple areas, with few shared boundaries or characteristics, or withdiscrete boundaries from one another, may share one identifier tag ordifferent instances of the same identifier tag. The system may performan efficiency analysis, estimating all tags that may be used based onpotential fixed dynamic elements and fixed elements, and estimating theprocessing cost while maximizing representational enhancement andverisimilitude, and factoring one or more of the outcome of suchanalyses into the decision whether and how many representational andother identifier tags to create and other data and aspects to associateor bring to bear on an output. The outcome of such analyses can bere-run periodically during an image capture activity, or can beperformed after image capture and recording in conventional media. Theoutcome can be changed, or the setting for the outcome can be altered,based upon user settings, which may grant a preference for more or lesselements or more or less enhancement generally, and for certain aspectsof the image captured. For example, the user may seek to boost localcontrast, as in high dynamic range (“HDR”) imaging, by boosting theamount of the system's fixed representational element generation withdifferent exposure characteristics and use Seed Data producing a highercontrast. The same can be done to increase or decrease any image aspectsor effects, to introduce new elements not originally in the imagecapture, recording and distribution, to create filters, and to providenew interrelationships between representational elements and parameters.

To illustrate, as the motion capture activity continues in FIG. 7, andrepresentational and other elements, as in FIG. 8, continue to becreated as decided by the user or in the system's analysis, the car 711collides with a hydrant 713. While the car and its shadow may haveinitially been defined by a fixed element identifier tag (or tags, forexample, one for the shadow having a less opaque aspect and higherz-index than the road element 809, while another being more opaque todefine the body of the car, other more metallic elements with sheenbeing tagged to have a reflective aspect, as one example), a new layout(using dynamic templates, or a series of templates, or tags that arethemselves changing both in number and kind (hereinafter “morphingtags”) may be created to capture the scene for distribution,manipulation, recreation and further redistribution. The initialidentifier tag 811 may shift on the 2-D plane of the user interface, andassociated data and relational aspects concerning its shiftingperspective or deformation may begin to be incorporated on the templateline or function with respect to that identifier tag. Alternatively, orin conjunction, new tags for depicting the distortion and deformation ofthe car as it impacts with the hydrant may be added, with RelationalAspects concerning the remainder of identifier tags describing otherrepresentational elements—for example, a collision relational aspectwith an identifier tag representing and constructing a representationalelement based on the hydrant 713. As discussed previously, physical anddescriptive models, functions, integrals, derivatives and otherequations may be associated in identifier tags, such as those createdfor the car and hydrant. For example, an anchoring function for thehydrant, as it is connected to a pipe below, and as both are 3-Dsubjects creating 2-D or 3-D imaging and representational elements, anda rupture point, after which water begins to rush out, requiring thecreation of dynamic water element(s), such as the blue and white dynamicrepresentational element 808, may be created. Component seeding andborrowing may be used to rebuild a “water rushing” texture onto theflowing water for example, by borrowing and applying seed analog datafrom fiber optic cables conveying water rushing at a geyser, as will beexplained in greater detail with respect to FIG. 10. Such a texture maybe blended onto a more basic image for the rushing water contours, withenhanced contrast, creating a hyper-realistic effect.

Any tag created by the system may have physical model characteristicsfor a computer-generated (“CG”) subject created by the system and, assuch, any number of CG or CG-associated elements may be generated intheir place by the system, automatically, or at the option of the user.With multiple views and/or logic, 3-D and physical characteristics maybe assigned as a subject of an element (and its identifier tag in thesystem). For example, the car element might be deduced to be roughly arectangular box in its base shape and moving, and occupying a particularspace with measurements. By associating a different element (forexample, an image or CG object representing a locomotive train) in thesame orientation as the original captured or channelled element, adifferent element can thereby be interposed. As more views of arecognized element are taken in the capture activity, a more thoroughimage element or 3-D model of its subject may be gradually built,leading to greater flexibility, requiring less system logic, to createresulting images with similar characteristics. But system logic may beparticularly helpful where a full capture of a particular element orbackground, in the instance of interposing images onto a background, isnot possible. In this way, distinct advantages over existing “greenscreen” technology (even setting aside the more limited scope ofapplication) can be seen.

FIG. 9 is a graphical depiction of an example resulting image or pointin an image stream as might be rendered by the present invention, fromthe capture activity depicted in FIG. 7. In addition to creating someaspects of the captured scene depicted in FIG. 7, it should be notedthat additional aspects (such as sourced data, textures, vibrancy, orother image aspects) have been incorporated or allowed to impact someimage elements. For example, the sun 705, now has a color or texturechange, which may be from Element Seeding or Borrowing, for example,from centralized analog data of a star surface that is stormier or moredramatic than the actual scene from the sun. Alternatively, a functionmay be applied by the system for varying aspects such as textures orother associated data in digital or analog format, or for controllingsuch data from ambient or seed and borrowing sources (e.g., settingluminosity, brightness or other limits, e.g., for peaks in variation)alone or in combination with information from the seed source (e.g.,observed variation information is plotted into a function or otherapproximation, or function or approximation varying the function orapproximation). The resulting blended, overlaid or otherwise affectedrepresentational element, 905, is depicted in FIG. 9 as darkened solidblack to represent that aspect in the image or point of an image stream.Similarly, representational element 908 may be a composite or hybrid ofimage data captured from the scene in FIG. 7, and the result ofapplication of moving texture data that is conditioned and applied froman external source, such as a geological geyser, depicted in FIG. 10,discussed below. This difference in the final image or point in an imagestream rendered is depicted by a darker line around the rushing waterrepresentational element 908 in FIG. 9.

FIG. 10 depicts an exemplary element seeding or borrowing source, andcapture activity involving the same, as might be used in accordance withaspects of the present invention and—specifically, a seed dynamictexture source geyser 1001, as might be used in the image capture andgeneration series depicted in FIGS. 6-8. In the example provided, arushing water texture source is sampled or taken from a geyser 1001 in ageologically active park (for example, Old Faithful at YellowstoneNational Park in the United States). A live streaming image of the park,which may be analog or directly transferred light, is taken by a cameraor other image or sensory data transfer device, 1003, (which may includefiltered or other selected data gathering) and made available forenhancing representational elements as digital or analog composites orhybrids by aspects of a system in accordance with aspects of the presentinvention. Element Seeding and Borrowing may be controlled withcentralized filing systems or libraries, with dedicated hardware andsearch indexing for higher-speed identification, matching andtransmission to any validated request (for example, with properauthenticated licensing) (not pictured) which allocate streaming, analogand direct image data, as needed for such Element Seeding and Borrowing.The system may stream or directly transfer such element seeds orborrowed items to enrich the images for which Element Seeding orBorrowing is requested, and may validate the seeding and borrowing.

Prior to applying the sampled or transferred element seed, that seedelement may be conditioned and altered to create more realistic effects,more limited or altered aspects, or more appropriately blend them withother representational elements. For example, the light source, 1005, isshifted to the left in the streaming or other data from the geyserdepicted in Element Seeding or Borrowing 1001. As such, the system orseparate seed library manager may first create a 3-D object frommultiple views of the object, and then shift perspective or lightsources virtually to match the light source perceived in the scene fromFIG. 7, or may simply partially flip horizontal axes of copies of theseed data (or in the scene) and blend images to create an overallimpression of an identical angle in the light source.

FIG. 11 is a depiction of part of a graphical user interface that mightbe used by a user implementing certain aspects of the present invention.As shown, there is a Cartesian plane display section 1101, allowing theuser to create and control the layout of identifier tags and incorporateor manipulate other data, aspects, elements, Relational Aspects,Interactive Parameters or resources in accordance with various aspectsof the present invention. (For convenience and to enhance discussionwith a familiar project, some of the same identifier tags previouslydiscussed with respect to FIG. 8, are shown in display section 1101). Byaccessing “Save” sub-menu items under the “File” menu, 1103, orotherwise, the user may save templates capturing such layouts, data,interactive parameters, work-flow, Element Seeding or Borrowing, and anyother image or image stream aspects of the present invention, including,but not limited to, identifier tags (such as the identifier tag forsemi-dynamic black element 1105) and their functions, integrals,parameters, characteristics, Relational Aspects, physical models,derivatives for capturing and rendering images or image streams. In apreferred embodiment, “windows” (such as that depicted as 1107), menuitems or sub-items (such as those depicted within sub-window 1109) and apointing cursor (such as that shown as 1111) are used by a user toinstruct the system and aid in carrying out aspects of the presentinvention, as described below. It is understood, however, that userinput through a user interface is not necessary to carry out aspects ofthe present invention, systems of some of which aspects may performfunctions instead by the system alone in response to receiving orgathering input (which may or may not be digital data) into a memoryand/or processor.

For example, in one potential aspect of the invention, the user maycreate a new identifier tag by moving the cursor to hover over andclicking on (“click on”) the menu heading “Elements” 1113 and selectinga sub-menu item titled “New Element” (not pictured). Optionally, theuser interface might prompt the user to set parameters affecting aspects(such as more basic or necessary aspects of the tag to be created, suchas its size, and whether it is to be dynamic (relating to a dynamicelement in the image to be created)). But the user (provided adequatepermissions and interface tools) generally may resize or define theimage or other data source, aspect, relationship or attribute even aftersuch a tag is created, with other transformational tools, such as thosepictured as click menu 1115, which may appear when the user clicks on anelement identifier tag. Preferably, the user would access such a menuwith a right mouse button, or other alternate mouse button click, ordouble-click, while restricting single primary mouse button clicks toallow repositioning, resizing, and other such aesthetic layouttransformations. Once executed, the “New Element” command stream mightcreate a visible box on the Cartesian work space, such as the boxmarking identifier tag 1105. Other shapes may be used, however,especially to indicate element parameters, attributes, aspects andboundaries and transformations, Interactive Aspects and RelationalAspects commanded by the user.

Turning to the click menu 1115 in more detail, various aspects of a userinterface can be understood. First, the user may set the nature of theelement tag as a Dynamic Element or Fixed Element through subpanelcontrols 1117-1123, and, if set as a Dynamic Element (via clicking oncontrol 1117, which indicates if selected by a check-box), may associatesource information, such as an image library that will be dynamicallymatched with an associated database to create an image, series or imagesor image stream, using drop-carrot selection control 1119. Preferably,the drop carrot window would then change its appearance to indicate suchlibraries and sources selected. If the user selects a Fixed elementsource using control 1121 (which also indicates if selected), he or shemay further specify a particular source for the fixed image element tobe created by the tag, using source selection drop carrot 1123. Thesource selection tools may allow the user to navigate a file storage orother operating system aspect, to locate sources, which may be inside oroutside the system, image formatted or streaming or directrepresentational or optical or analog in nature, among other things, andmay include Element Seeding or Borrowing. The user may set variousRelational Aspects using click menu 1115, including but not limited tothose pictured, such as layering of representational elements viacontrols 1125-1129, to be created by identifier tags, including the onesubject to the particular click menu 1115, summoned by the user. Forexample, by selecting the manipulation of the “z-index” of an associatedelement, using controls 1125 and 1127, and indicating a number (in thisexample, the z-index number 72 is typed in by a user) the subjectelement will be placed in a layer above any element with a z-index of 71or lower, but will be placed below any element with a z-index of 73 orhigher in a generated image, set of images and image stream. It isunderstood that, in other aspects of the invention, any RelationalAspects may themselves be dynamic, changing with a function, such as atime function, or the status of other Relational Aspects impacting theelement(s) subject to the tag. For example, through stacking control1129, the element(s) subject to the tag may be directed to carry az-index following or leading another element such that, if that otherelement changes its z-index, the identifier tag 1105 will change with orchange the z-index of the “stacked” element accordingly. In anotherexample of Relational Aspects that may be set, a pinning anchor position(in this instance, the element is set to anchor at the most upper-leftpoint(s)) and pinning preference (in this instance, a pinning preferenceof 7, such that any colliding or otherwise conflicting element with alower index number will cause the element(s) or subject(s) associatedwith 1105 to move and/or deform) may be set forth for the element usingpinning controls 1131 and 1133. Using Subject Relational Aspect tools1135 and 1137, the user may define some aspect or object (real or invirtual space) that may assist in generating or renderingrepresentational elements using aspects of the present invention. Forexample, by selecting a 3-D source with 3-D source tool 1135 (indicatedif selected by, for example, an html check box) the user indicates tothe system that aspects of the element will be rendered and generated(including aspects affecting other elements and the image, set of imagesor image stream to be composited and/or generated) according to a 3-D orotherwise multidimensional subject. For example, by further selecting aphysical model, with physical model selection tool 1137, a user may, forinstance, select a red ball as a 3-D subject, or bouncing red ball as a4-D subject (including space-time, not just 3 Cartesian coordinates),and, if a 2-D Dynamic Element is to be generated from such a subject, apoint in an image stream may, for example, display a shaded circleaccording to those 3- or 4-D characteristics at the given perspective,time and with the impact of other Relational Aspects.

Various sub-selections within 1137, with physical models forinterrelational aspects are possible (“Linked to objects” being shownwhich would permit the borrowing of physical behavior or characteristicsfrom another element or object), including the definition of objects'affects and impact on one another. In addition, the system may match ordeduce environmental information or seed or borrowed data forestablishing all or part of a physical model. For example, if an objectis defined by a model providing magnetic, gravitational or otherpotential force-delivering effects, a function describing those effectsand leading the tag to represent associated elements according toequations and functions dictating subject object behavior, accountingfor interrelationships with other objects according to the physicalmodel. Every know physical effect, force and object and energy principlemay be used in descriptive approximations and/or equations of varyingrealism and processing budgets or from Element Seeding and Borrowing,which may be without encoding or translation, and directly transferred.As discussed previously, Element Seeding or Borrowing may take direct,real injections of image sources or data sources of similar or relatedobjects behavior in the real world, in a dynamic, indexed search andmatching system to match objects with real-life or more complexapproximation resource libraries. In this way, centralized, potentiallyremote systems, can stream more accurate texture, physical model,shading and other data, with less impact on the central processor of thesystem, or client-side terminal, to many similar systems. Alternatively,a local source for a viewing system may apply such Element Seeding orBorrowing, preferably in parts simply used to provide real texture ormovement randomness (such as in Brownian motion) into the image, imageseries or stream of generated images.

Regardless of whether Element Seeding or Borrowing is used, andregardless of whether the system processor runs programs within thesystem for generating and applying physical models to elements managedby identifier tags, the physical models are preferably those with thegreatest verisimilitude and lowest processor work requirements. Forexample, while Newtonian and Relativistic equations can be highlyaccurate as part of physical models, Newtonian equations can be complexas applied and inaccurate, particularly in the aggregate, andRelativistic equations require a great deal of processing power.Preferably, a generalized space occupying inter-collision model may beused as a physical model, with different objects, strengths and momentumvectors described (hereinafter, the “Unified Model” model). In such aUnified Model, it is taken that all forces, including gravity,electromagnetism and the weak and strong nuclear forces, are simply thenet or secondary effect of objects or waves with momentum or inertiaoccupying space and colliding, rubbing or passing through one another,and transferring energy in the process. Preferably, in a Unified Model,gravity, electromagnetism and the weak and strong nuclear forces arethemselves a shadowing effect from ambient, abundant or ubiquitousradiation or moving particles. In the instance of gravity, theinter-colliding particles and/or waves may be represented or objectifiedto travel at or near the speed of light, which may allow the ubiquitousconcentration of such EM or particles in different reference frames—aswith the ubiquitous universal background microwave radiation, which isrelatively uniform in all frames of reference despite differingvelocities through space. Thus, a Unified Model describes forces appliedat a distance as actually the effect whereby two or more objects blockambient activity from one another, with the net effect of greater forceon the outside surfaces than the inside surfaces, which are protectedfrom a greater concentration of colliding waves or particles occurringoutside of the pair of objects in space, one object shielding the other,and causing an overall attraction effect.

Whether or not a Unified Model is actually the cause of gravity andother unexplained extant physical forces, it can be more efficient forthe system to use one concept, of element collision and physicalinterference, rather than several functions with differing constants(such as wind resistance and collision forces for some objects andtensile strength based on the colligative nature of covalent bonds orthe gravitational constant G in the well-known Newtonian equation forgravity, F=GMM/r²), for others.

Sub-panel control 1139 of click menu 1115 controls the opacity ofassociated elements, (Relational Aspects of identifier tag 1105). Set to100% opacity, as pictured, any elements placed in an area overlappingwith the associated element in display section 1101, with a lowerlayering z-index, will be blocked from direct view in a resulting image,set of images or image stream in that overlapping area. If anothersetting is selected for opacity control 1139, such as, for example, 50%opacity, the evenly distributed amount of light relative to 100% will bereduced accordingly for the element, while permitting the remainder toemanate from the next element below (if itself 100% opaque and, if not,that residual amount of light from lower layers as may pass through).With or without a fraction of light emanating from lower elements,various blending modes between overlapping layers may be selected, suchas “overlay” (pictured, which, as one example, requires bothmultiplication and division of color and shading components in the sameoverlapping location of the different layers) with blending mode control1141. In addition to the typical blending modes available in much imageediting software, a sampling routine may be used to approximate colorand shading levels and adjustments that are required from smaller area(less than the whole of the element produced) to smaller area (forexample, sizes of the user's choosing) within the overlapping area toget blend effects similar to what would occur with digital and pixelatedsources, when using analog and direct image sources. In addition, theblending mode, or other such Relational Aspects, may be set to vary overtime or according to some function, with Dynamic Function control 1143.Yet other examples of possible Relational Aspects are provided inReflection Source control 1145 and Reflection On control 1147, which, ifenabled, permit light emanating from or received by an object orelement, respectively, to reflect light. Some controls, such as externallight source control 1149, permit the introduction of additionalelements, such as a light source, and may permit the user to create thedirection, space distribution and elevation of unlimited light sources,as with light direction(s) control tool 1151, or even allow directsimulated photon vector patterns distributed in space, which would notbe possible with real light sources (for example, in a 3-Drepresentation of such vectors that includes the interaction ofboundaries of an element or subject object). As mentioned elsewhere inthis application, it is within the scope of the invention that anyelement, aspect, data source, Relational Aspect, source object, andother thing that may be set by the system, and identifier tags inparticular, may be done in multiple instances and sets, as by the “Add”html button 1153, which allows the user to add an additional lightsource, and would create a new panel to the user interface, with a newexternal light check box control and directional or 3-D locationindicator for the light source (not pictured). Text control sub-panel1155 of click panel 1115 allows the user to add a text element to becontrolled, conditioned and placed by the accessed identifier tag, 1105,in this instance. The text may either be entered directly, by clickingon text entry space 1157, or user-variably keyed to a source, such as adatabase or streaming source, which may be made for creating a fixedtext element (e.g., by clicking a fixed text selection tool 1161) or adynamic text element (e.g., using dynamic text selection tool 1159, andselecting a column with the desired textual cell contents (selector notpictured) for each image, image within an image set or point in an imagestream). Various conditions, layouts and effects may be added to thetext, including Relational Aspects that allow the text to be oriented inany spatial configuration relative to another element. For example, theuser may select that the text be oriented to the upper-right of theelement, as pictured, with orientation selector 1163, or may dictate adirection for the text to lean, pitch or yaw (which may include 3D texteffects) as with lean control 1165. With a pixel margin selector, 1167,the user may further dictate a distance between the text and the elementto be created, in addition to the given angle and orientation, and, byusing negative numbers, may place the text directly over another elementcontrolled by the identifier tag. The text and any associated otherelement may have any relational aspect between one another as they mayhave with other elements. Justification and alignment controls such as1169, Bold, Underline and Italics buttons 1171 and font size selector1173 are among the text parameters and conditions that may be applied tothe resulting text.

As discussed elsewhere in this application, any Relational Aspect set bythe user may vary over time or over a function (such as interdependenceon the activities or existence of other elements or subject objects, asin a physical model). The physical models themselves or other RelationalAspects may be set to change over a time or other function by DynamicFunction control 1143.

It is understood that, in addition to impacting one another, identifiertags and their associated elements may cause the creation or removal ofidentifier tags, including multiple identifier tags, in some instancesmultiplying, dividing and creating or deleting existing tags by othermathematical functions, as dictated for efficiency and realism by thesystem to add pixel, or similar block elements, with greater resolutionin particular regions, especially dynamic element intensiveregions—which may be implemented by the system itself as well as by auser.

The creation of new elements may also be carried out with variouspainting and other image creation tools, such as paint brushes and otherdrawing, blending and interrelational aspect tools, a few of which areshown in tool panel 1175, including a pointing/movement selection tool1177, a paintbrush tool 1179, a color fill tool 1181, and a colorselector 1183. Each of these tools may be used either in the Cartesianspace for design (pictured as 1107) or in alternate views. In this way,accessing and creating tags may be more user friendly, especially inconjunction with a preview window view mode (not pictured), accessible,for example, by clicking on the “Preview” menu item 1185, which wouldpresent the elements to be created, or visual approximations thereof, tobe painted or affected with instantaneous image appearance feedback forthe user. To illustrate the state of dynamic elements in such a viewmode, a dynamic symbol or symbols may be used, or a time or functionselection or scroll tool (not pictured) would permit the user to summonthe element appearances or sound at a particular point or segment intime or other function, in an image set or stream (preferably, with anuser-optional opaque rendering of any movement path for the dynamicobject also pictured). Optionally, the controls discussed above could beactuated by “on screen” manipulation of elements both in the Preview andCartesian design viewing modes. For example, the user may be able tochange the light source angle by gripping and moving a light sourceelement, as well as elevating or lowering a control point out and intothe page, along a z-axis, with instantaneous viewing of the resultingdifference in light, reflections and shading in the preview view mode.

The user may determine when to generate images, a set of images, or animage stream with menu items, as well as dictate other administrativeand organizational tasks, such as output delivery location, databasesources and types, and move the contents of one database to another,using, for example, menu items such as some pictured as 1109.

FIG. 12 is a schematic block diagram of some elements of a system 1200that can be used in accordance with aspects of the present invention.The generic and other components and aspects described are notexhaustive of the many different systems and variations, including anumber of possible hardware aspects and machine-readable media thatmight be used in accordance with the invention. Rather, the system 1200is described here to illustrate how aspects may be implemented. Amongother components, the system 1200 includes an input/output device 1201,a memory device 1203, storage media and/or hard disk recorder and/orcloud storage port or connection device 1205, and a processor orprocessors 1207. The processor(s) 1207 is (are) capable of receiving,interpreting, processing and manipulating signals and executinginstructions for further processing and for output, pre-output orstorage in and outside of the system. The processor(s) 1207 may begeneral or multipurpose, single- or multi-threaded, and may have asingle core or several processor cores, including microprocessors. Amongother things, the processor is capable of processing signals andinstructions for the input/output device 1201, analogreceiver/storage/converter device 1219, and/or analog in/out device1221, to cause a user interface to be provided for use by a user onhardware, such as a personal computer monitor or terminal monitor with amouse and keyboard, including user interfaces discussed above, withrespect to FIGS. 2, 8 and 11. The processor 1207 is capable ofprocessing instructions stored in memory devices 1205 and/or 1203 (orROM or RAM), and may communicate via system bus(es) 1275. Input/outputdevice 1201 is capable of input/output operations for the system, andmay include innumerable input and/or output hardware, such as a computermouse, keyboard, networked or connected second computer, camera orscanner, mixing board, real-to-real tape recorder, external hard diskrecorder, direct image transfer or analog devices, additional movieand/or sound editing system or gear, speakers, external filter, amp,preamp, equalizer, computer display screen or touch screen. Such adisplay device or unit and other input/output devices could implement auser interface, such as those discussed above, in reference to FIGS. 2,8 and 11. 1201, 1203, 1205, 1207, 1219, 1221 and 1223 are connected andable to communicate communications, transmissions and instructions viasystem bus(ses) 1275. Storage media and/or hard disk recorder and/orcloud storage port or connection device 1205 is capable of providingmass storage for the system, and may be a computer-readable medium, maybe a connected mass storage device (e.g., flash drive or other driveconnected to a U.S.B. port or Wi-Fi) may use back-end (with or withoutmiddle-ware) or cloud storage over a network (e.g., the internet) aseither a memory backup for an internal mass storage device or as aprimary memory storage means, or may simply be an internal mass storagedevice, such as a computer hard drive or optical drive. Generallyspeaking, the system may be implemented as a client/server arrangement,where features of the invention are performed on a remote server,networked to the client and made a client and server by software on boththe client computer and server computer.

Input and output devices may deliver their input and receive output byany known means, including, but not limited to, the examples shown as1209, 1211, 1213, 1215 and 1217. Because the images managed, manipulatedand distributed may be any representational or direct impression ortransfer captured from any activity, any phenomenon that may be sensedmay be managed, manipulated and distributed may be taken or converted asinput through any sensor or carrier known in the art. In addition,directly carried elements (for example a light stream taken by fiberoptics from a view of a scene) may be directly managed, manipulated anddistributed in whole or in part to enhance output, and whole ambientlight or or other phenomena information may be taken by a series ofsensors dedicated to angles of detection, or an omnidirectional sensoror series of sensors which record direction as well as the presence ofphotons, compression waves or other directional and present phenomenarecorded, and may exclude the need for lenses (or ignore or re-purposesensors “out of focal plane” for detecting bokeh information orenhancing resolution as focal lengths and apertures are selected), onlylater to be analyzed and rendered into focal planes or fields of auser's choice through the system. For example, a series of metallicsensor plates that resonate with photons propagating in particulardirections would also be capable of being recorded with directionalinformation, in addition to other, more ordinary light data recorded bysensors. While this example is illustrative, it is understood that anyform of electromagnetism, compression wave or other sensory phenomenonmay include such sensory directional and 3D locational information,which may also be made possible by multiple locations of sensing,preferably, in a similar, if not identical, time frame. Through angularanalysis of photons originating from a common source element, distancesand 3-D contours can be measured and included for elements and elementfeatures, and used to develop 3-D and 4-D physical models used by thesystem. The system may condition, select all or part of, alter and/orgenerate composites from all or part of such direct or analog imagetransmissions, and may combine them with other forms of image data, suchas digital image files, if such direct or data encoded sources are used.

While the illustrated system example 1200 may be helpful to understandthe implementation of aspects of the invention, it is to be understoodthat any form of computer system may be used—for example, a simplercomputer system containing just a processor for executing instructionsfrom a memory or transmission source. The aspects or features set forthmay be implemented with, and in any combination of, digital electroniccircuitry, hardware, software, firmware, or in analog or direct (such aslight-based or analog electronic or magnetic or direct transmission,without translation and the attendant degradation, of the image medium)circuitry or associational storage and transmission, as occurs in anorganic brain of a living animal, any of which may be aided withexternal detail or aspect enhancing media (for component seeding andborrowing) from external hardware and software, optionally, by networkedconnection, such as by LAN, WAN or the many connections forming theinternet. The system can be embodied in a tangibly-stored computerprogram, as by a machine-readable medium and/or propagated signal, forexecution by a programmable processor. The method steps of theembodiments of the present invention may be performed by such aprogrammable processor, executing a program of instructions, operatingon input and output, and generating output. A computer program includesinstructions for a computer to carry out a particular activity to bringabout a particular result, and may be written in any programminglanguage, including compiled and uncompiled and interpreted languagesand machine language, and can be deployed in any form, including acomplete program, module, component, subroutine, or other suitableroutine for a computer program.

FIG. 13 is a process flow diagram depicting exemplary steps 1300 thatmay be carried out by a control system such exemplary control system1200, discussed above, or combination of control systems, implementingexemplary programming, methodology and other aspects of the presentinvention. Beginning with step 1301, the system takes in at least oneimage as input (for example, a jpeg or a series of video frames) andidentifies or defines at least one Dynamic Image Element, which is anapparent form or structure within the image that is depicted as movingor otherwise physically changing over time. The system then proceeds tostep 1303, in which it determines whether embedded, associated orotherwise related physical model data describing the behavior of theDynamic Image Element has been provided as input (for example, as a partof or along with the image). In some aspects of the invention, aspecialized new image file format, containing at least one embeddedphysical model, subject object data and data defining or aiding indefining the Dynamic Element, in addition to more conventional imagedata, may be received by the system as input. An example of such aspecialized file format is discussed in greater detail, below, inreference to FIG. 16. But in other aspects, a separate file or data feedcontaining physical modeling data may be provided as well, or instead ofsuch a file format. If physical model data relevant to the Dynamic ImageElement has been received as input, the system then proceeds to step1305, in which the burden of utilizing the provided physical model,subject object data and/or definitions so provided as input areevaluated by the system. More specifically, the system may make anassessment or projection of the cost in terms of its resources(including the cost of storing that input data, the speed with which itcan be accessed, transmitted and used to render images). Afterperforming that assessment, or if no such data has been received in step1303, the system then proceeds to step 1307.

In step 1307, the system next performs a similar efficiency assessment(or accesses a record of such an assessment) of at least one of aplurality of physical models and/or subject object data accessible bythe system to aid in representing or simulating the Dynamic ImageElement in a new, composite image. They system may create an efficiencyassessment score, representing a ranking of such physical models anddata accessible by the system for that purpose. The system then proceedsto select physics models with the highest score—e.g., the least cost(most efficient) to system resources, and/or highest rendering andtransmission speed (without an unacceptable loss in image quality)within a range of competitive high scores, in step 1309. In someembodiments, the physical model with the highest tentative score at thisstage may be stored and associated with the Dynamic Image Element and,if the image is accessed for viewing by a user, that model will be usedto aid in rendering the image on viewing hardware of the system. If thesystem selects physical models other than that provided as input for thepurpose of aiding in rendering the Dynamic Image Element, that input maybe discarded, not stored or associated with the element, or, if stored,deleted, or placed into a relatively remote data storage location, as abackup option for rendering the image. In any event, proceeding to step1311, the system may proceed to perform a more detailed or intensiveassessment of the tentatively selected physical models (if more than oneare selected), to more accurately and thoroughly determine theirefficiency scores, relative to one another, for aiding in rendering theDynamic Image Element, and selects the physical model with the highestefficiency rating (with an acceptable image rendering quality, and/or inan algorithm including a weighting for image rendering quality).Following that selection, alternative physical models, if they have beenstored, encoded or otherwise associated with the Dynamic Image Element,may be overwritten, with data concerning the selected physical model, instep 1313. The system may then return to the starting position or, insome alternative embodiments, proceed to step 1315, in which itestablishes a link to aspects of the physical model within a tagassociated with the Dynamic Image Element (e.g., placed within aCartesian coordinate map or subsystem at or related to the location ofthe Dynamic Image Element within the Image). An aspect of the physicalmodel, or another physical model, related to higher quality imagerendering may also be so linked, and allow the system to improve imagequality if enough resources are available to do so, in step 1315.

FIG. 14 depicts an image capturing activity, comprising a camera'sviewfinder frame 1402, framing a scene 1400 that includes a dynamicimage element (namely, a ball 1401, rolling down an inclined ramp 1405),and some exemplary forces and resulting movements that may be capturedand rendered by a system and method managing images in accordance withaspects of the present invention. Within scene 1400, ball 1401 is pulleddownward, toward the Earth and bottom of FIG. 14, by gravity—as shown byforce of gravity vector arrow 1403. However, ball 1401 does not begintraveling directly downward with that force, because it encounters anormal structural force, from the surface of ramp 1405, andperpendicular to it. That normal force is shown by normal force vectorarrow 1407. Normal force 1407 opposes and matches the magnitude of thevector component 1409 of gravitational force 1403 that is directed intothe surface of ramp 1405, as an equal and opposite reaction to it, whichcan be described by Newtonian mechanics. A remaining force component offorce of gravity vector 1403, as illustrated by vector arrow 1411,remains, unopposed by ramp 1405, and pulls ball 1401 along the plane oframp 1405, to the left and downward, in the perspective of the figureand frame 1402. Opposing that pulling force is an opposing force offriction (or stiction), shown by force of friction vector arrow 1413.

A complex, resolved or net force may remain despite that opposing force,and ball 1401 may begin to move downward and to the left, accumulatingkinetic energy. A system using camera frame 1402 to capture scene 1400can assess the direction of gravity in the scene, and recognize ball1401 and inclined plane 1405 as objects, and, employing a physics engineand physical model incorporating Newtonian mechanics, can assume thepresence of and record data concerning each of the forces and vectorarrows discussed above, to create a physical model of the scene.Furthermore, if ball 1411 begins to roll downward and to the left, inthe manner or within a tolerance range of the motion projected by thatphysical model, the system can confirm the validity of the model, andupdate it for more complex physical force components that it had notanticipated (e.g., an initially unassessed force of wind 1417, forexample, from fan 1415 within the scene or from complex textures on thesurface of ball 1401 or ramp 1405). Although a limited number of suchforces are shown, it should be understood that a much greater, and morecomplex, and subtle, set of forces may exist, impacting the motion ofthe dynamic object 1401 and a resulting dynamic image element from theimage capturing activity, and the forces shown and simplicity of thescene are for illustrative purposes only. But even within the simplified“sandbox” of the scene depicted, the potential forces beyond what hasbeen illustrated with vector arrows are many and varied, and sufficientto illustrate further aspects of the invention.

FIG. 15 depicts an image capturing activity, comprising the same cameraviewfinder frame 1400, and a similar scene 1500, with dynamic object1401, and a force vector 1517 resulting from a new physical modelselected and applied by a system capturing and rendering dynamicelements for image management in accordance with aspects of the presentinvention. Rather than assess and create each of the forces and vectorsset forth above, with reference to FIG. 14, a control systemimplementing aspects of the present invention uses a lessresource-intensive, otherwise more efficient, or faster-executedphysical model to analyze and record the movement of a dynamic imageelement 1501, corresponding with ball 1401. More specifically, and asdiscussed elsewhere in this application, the control system may assess avariety of alternative physical models and physics engine options (forexample, from a system library of such available models and options)that would describe the apparent motion of ball 1401 and resultingdynamic image element 1501, and select one that most efficiently doesso, while maintaining resolution and other realism standards.

For instance, in the example depicted, the control system has selectedand recorded a collision-based physical model, with a virtual object1519 of equal size and weight striking ball 1401, and causing ball 1401to have a net force downward and to the left-hand side of the figure, inthe same direction as the net resolved force discussed above, withreference to FIG. 14. As a result, the control system must only recordand implement a single, less complex physical algorithm, to accuratelyrepresent the movement of dynamic image element 1501, than with thecomplex physical models with the resolution of innumerable force vectorsdiscussed above. As the scene progresses, and ball 1401 experiencescontinued acceleration due to gravity, additional collisions, withadditional virtual objects, such as exemplary second collision object1521, may be hypothesized and recorded to describe the apparent movementof ball 1401 as a dynamic image element 1501. Second collision object1521, as with object 1519, is a virtual construct of the selectedphysical model and, as such, need not be physically depicted noraccounted for after collision with a virtual subject of dynamic imageelement 1501. Thus, although second collision object 1521 is shown atsome distance from first virtual collision object 1519, and travelingtoward it with a velocity vector 1523, it will not be modeled by thesystem as colliding with object 1519, and instead will pass through thevirtual space where object 1519 had been during and after collision withthe virtual subject of dynamic image element 1501. Other aspects ofphysical collision models, such as deformation and heating, can also beomitted in the simplified, efficient physical model selected andimplemented by the control system, with the goal only or representingthe motion of ball 1401 accurately by creating the motion of dynamicimage element 1501 as that described by the selected physical model. Anumber of such colliding virtual objects may be created, recorded andapplied by the control system, to reach a realistic portrayal of themotion of ball 1401 as captured by the camera. The number of suchvirtual objects, and the timing of their collisions with a virtualsubject of dynamic image element 1501 may be selected depending on theframe rate of the camera, and stored image format, such that element1501 appears in the place observed by the camera at intervals matchingthe instance of each frame. Thus, the distance and velocity pictured forvirtual object 1521 is illustrative only, of the wide variety ofpossible virtual objects, the timing of collisions or other physicalinteractions, and the energy transfer from such objects or other modeledphysical phenomena, that may be selected and applied by the controlsystem to depict observed changes in objects with dynamic imageelements.

FIG. 16 is a schematic outline of several components 1600 of a new imagefile format, with embedded physical model data for managing virtualobjects as subjects of corresponding depicted dynamic image elements. Itshould be understood that the file format components set forth in thisfigure, and their order of occurrence, and the recitation of some moreconventional image file components, are exemplary in nature. A widevariety of different component divisions, orders and surrounding fileformat features may be used, in addition or as an alternative to thatpictured, while carrying out aspects of the present invention. The firstfeature of the file format may be a set of file headers 1601, settingforth basic parameters for identification and treatment of the file, andfile type. For example, the type of file may coded “PHI” in the headers,signifying that it is a Physics-model enriched Image, to operatingsystems, image editing software and software engineers accessing thefile and headers. Component 1603 may be included in some embodiments ofsuch a file and file format, and includes coding for data concerningoriginally-embedded physics models (at the time the file was firstcreated) to aid a computer system rendering dynamic image elementswithin an image stored in the file format. Those data may include, forexample, algorithms, equations, virtual object data, or sets thereof, aswell as access links to aid the system in accessing additionalcomponents of a physics model, physics engine or virtual objects storedat other locations within the system or an accessible network (e.g., theinternet) to aid in rendering dynamic image elements of the imagecaptured in the file. These data may also include efficiency, speed andresource usage ratings relevant to the use of the original physicsmodel, allowing the system to perform comparisons to other physics modeloptions, in accordance with aspects of the invention set forth in thisapplication. Component 1605 is also included, and comprises similar datarelated to the currently system-selected and active physics modelsaiding a computer system rendering dynamic image elements within animage stored in the file format. Such selections of physics models andvirtual object data may be made in accordance with any and all of thetechniques set forth elsewhere in this application. Component 1605 mayalso comprise coding indicating to the system that the data set forthwithin the component is the currently selected, active data forphysics-based dynamic image element rendering. In addition, anotheroptional file component, 1607, may be included, which sets forth similarinformation for previously selected or active physics model data, inpreviously saved versions of the file, as well as a timeline or otherversioning history indicating the selection criteria and other relevantfactors leading to and impacting its previous use.

File components 1600 may next include a series of data chunks directlyrelated to image appearance when rendered by a system processing thefile and displaying it (e.g., on a computer monitor), in component 1609.As with image data chunks in other image file formats (e.g., the PNGformat) the chunks within component 1609 may include bitmap or otherraster data or vector data) or data related to frames, framerate (forvideo files with multiple frames) or bitrate, among many otherpossibilities. As a different physics models, and subject virtualobjects may be used by the system to create different versions of theimage file, in accordance with aspects of the invention set forth inthis application, more, less or modified versions of such chunks may berequired to ore accurately and/or more efficiently render the image. Forexample, if more of the physical appearance of the image rendered isdictated by a newly-selected and stored physics model, less directlydescriptive raster or vector data may be required to describe and renderthe image. Accordingly, in an optional component 1611, a historicalrecord of the deletion and/or addition of different chunks may berecorded, enabling the system or a user to dial back or resort toprevious chunks, for example, by accessing an external record of themvia a network link (a.k.a., an “Out-Reference.”)

Proceeding to component 1613, the file format may also comprise datarelated to virtual objects employed by the currently-selected physicsmodel(s) used in the file to render dynamic image element(s). The filemay also contain, within this component, a reference for each suchobject to the relevant physics model data used in conjunction with theobject to render the appearance of a dynamic image element when the fileis processed by a control system. Components 1615 and 1617 may containsimilar data for objects, and referenced physics models, originally orotherwise historically used, in previous versions of the image file. Inlieu of more complete object data, however, simpler data defining oridentifying the virtual object may be set forth (for example, withexternal links to better define or restore previous versions of thefile, or historic image rendering facility, as may become necessary tobetter render the file under different circumstances, such as with a newrendering system, or to repair later, corrupted data.)

The file format may also contain a number of ancillary components anddata, as may be used in wide variety of different image file forms, incomponent 1619.

I claim:
 1. A system for increasing efficiency and realism in computermanagement of digital images comprising a first processor and anon-transitory machine-readable medium, which system receives at leastone image as input and defines or identifies at least one of any DynamicElement(s) within the at least one image, and associates at least onephysical model relevant to describing at least part of some activity ofsaid Dynamic Element(s) by encoding data related to said at least oneimage and said physical model through the use of said processor and saidmachine-readable medium; and wherein said system creates a new compositeimage incorporating tags to simulate said at least one aspect of oneDynamic Element(s), which tags are within a Cartesian coordinatesubsystem of a graphical computer display comprised in said system; andwhich tags are encoded by said first processor for said first processoror a second processor(s) to access image elements and/or aspects andintegrate them into a new, composite image.
 2. The system of claim 1, inwhich the system defines or identifies at least one RepresentationalElement.
 3. The system of claim 1, in which the system defines oridentifies at least one image Element using Variation Toleration.
 4. Thesystem of claim 1, in which the system identifies at least one imageElement(s) by associating said tag(s) with them.
 5. The system of claim1, in which said at least one physical model is selected from aplurality of physical models available to the system.
 6. The system ofclaim 1, in which said at least one physical model is created by thesystem, by combining or applying at least one available model-buildingresource(s).
 7. The system of claim 5, in which the system determineswhether a physical model has already been associated with any aspect ofsaid image before associating said at least one physical model relevantto describing at least part of some activity of said Dynamic Element(s).8. The system of claim 6, in which the system determines whether aphysical model has already been associated with any aspect of said imagebefore associating said at least one physical model relevant todescribing at least part of some activity of said Dynamic Element(s). 9.The system of claim 5, in which the system compares the amount of spacerequired for storing data relevant to one of said physical modelsavailable to the system and the amount of space required for storingdata relevant to another of said physical models available to the systembefore associating said at least one physical model relevant todescribing at least part of some activity of said Dynamic Element(s).10. The system of claim 6, in which the system compares a systemallocation required for implementing at least one first model-buildingresource, with a system allocation required for implementing at leastone second model-building resource, and, based on that comparison,selects at least one model-building resource for use.
 11. The system ofclaim 5, in which the system compares the time required to render animage using one of said physical models available to the system with thetime required to render an image using another of said physical modelsavailable to the system.
 12. The system of claim 1, in which the systemincorporates or applies onto said new image an infinitely varyingelement from a source external to said new image; wherein the infinitelyvarying element is a source used to generate aspects of said new image.13. The system of claim 12, in which the infinitely varying element usesor comprises an analog image element or an image element directly formedfrom the emission of light from an imaged object or phenomenon.
 14. Thesystem of claim 1, in which the appearance of camera shake is reduced ingenerated image(s) by the system repositioning identifier tag(s) forDynamic Representational Element(s), and/or repositioning the generatedDynamic Representational Element(s) themselves, in a more constantposition in the generated image(s) than would occur without saidrepositioning identifier tag(s) and/or said repositioning the generatedDynamic Representational Element(s).
 15. The system of claim 1, whereinsaid physical model comprises at least one 3-dimensional subject for atleast one image element.
 16. A system comprising a first processor and anon-transitory machine-readable medium configured to create, modify oruse at least one tag configured to cause at least one image element(s)to be included in an image or image stream generated by the system,which image elements are implemented instead of, and simulate theappearance of, other elements of other versions of the image or imagestream; wherein said at least one tag is within Cartesian coordinates ofa graphical computer display comprised in said system; and said at leastone tag is encoded by said first processor for said first processor or asecond processor(s) to access image elements and/or aspects andintegrate them into a new, composite image.
 17. The system of claim 16,comprising a user interface configured to permit a user to create imageelements, and configured to permit a user to include associated data,Interactive Parameters or infinitely varying elements from a sourceexternal to said image or image stream with the at least one tag. 18.The system of claim 16, wherein the system is configured to incorporatea physical model comprising 3-D or 2-D subject as associated data withat least one identifier tag.
 19. The system of claim 18, wherein saidsystem incorporates at least one physical model selected from multipleavailable models, based on an efficiency or optimization rating for eachof said available models.
 20. The system of claim 18, in which saidselected model comprises simulating a movement of said 3-D or 2-Dsubject due to gravity, electromagnetism, the weak or strong nuclearforces, or another remotely-applied force, by applying a formula for atleast one collision or other contact force.